World of Video

home *** CD-ROM | disk | FTP | other *** search

/ World of Video / World of Video.iso / gfxprograms / animtools / videotracker / vtutil1.lha / Imploder_Doc < prev next >

Wrap

Text File | 1993-12-28 | 75KB | 1,796 lines

Peter van Campen: I had to put the manuals of this program in one file to save diskspace. *************************************************************************** This archive contains the Imploder 4.0 distribution. -All files directly related to the Imploder have been moved into the "Imploder" directory and subdirectories. The Imploder allows you to reduce the size of executable files while having them retain their full functionality. There are other "crunchers" or "packers" available for the Amiga, but none are as mindful of the complexities of your Amiga system as the Imploder (IMHO). In addition to this, its algorithms are more efficient, both in terms of speed, and size reduction. -The FInf drawer contains a versatile directory lister and file type recognizer. This one is bursting with options, many of which are sorely needed but not present in any of the other listing utilities I know of. Of course it recognizes whether files are Imploded or not. -The DImp drawer contains a disk-archiver that makes use of the Implosion algorithm. Special features include the creation of self extracting and explaining disk archives, support for any device with a floppy like track layout, and the flexible selection of cylinder ranges. *************************************************************************** How to operate the Imploder should be fairly obvious to the average Amiga user. So I'll try to highlight only those aspects of the user interface that aren't immediately obvious, and of course this document will explain how certain selections influence the processing behaviour. ------------------ The User Interface ------------------ ** Configuring the Imploder ** When you run the Imploder, either from the CLI or WorkBench, you'll notice the music and NTSC size screen. This setup is configurable by way of commandline switches or tooltypes. Note that, under Kick 1.3 or less, when one sets a tooltype using the WorkBench's info option, the appended "=" is required for proper recognition. The default setting is enabled music without modifications to the filter or NTSC/PAL mode. However, you can disable the music using the "NOMUSIC" commandline switch, or the "NOMUSIC=" tooltype. The lowpass filter can be disabled for less muffled sound by specifying "NOFILTER/NOFILTER=". To people with PAL Amigas, the shrunken NTSC screen might look ugly, however if there is a fat Agnus chip in your machine, it can be toggled from a 50Hz to 60Hz display refresh rate. Setting "NTSC/NTSC=" will cause the Imploder to do this whenever its screen gets upfront. In addition to this, there's the SHORTROOT configuration switch. This option is related to using the library. See the "library" document for further information on this. ** The File Requester ** Whenever there exists the need to specify a filename, a directory window will appear in front of the message window. Only a few of its options need clarification: -Switching on the ID toggle gadget causes the directory window to display the type ID's of all files. This is useful, for example, to check if files have already been imploded. This option will increase the time needed to access a directory though. -Clicking on the proportional scroll gadget on the left will cause the list of filenames to be sorted. -Switching off the .INFO toggle gadget causes the directory window not to display the *.info files used by the WorkBench's icon system. -Selecting a file is done by placing the mouse pointer on top of its name and clicking the left mouse button. In addition to the normal ways of confirming your selection, you can double-click on a filename. -Next to the immediately obvious ways of scrolling through a long directory, one can place the pointer within the filename window, press the left button, and while holding it down move the mouse above or below the filename window, try it; it works great once you get used to it. -One does not have to wait for the entire directory to load before selecting a file or entering a sub directory. ** Keyboard Equivalents ** Many functions selectable by mouse also react to keys. The menu options have the usual right-amiga+key shortcuts. In addition to this; - Hitting any key removes the about requester displayed during startup. - Use the escape key to exit the Imploder. - The "S" key starts processing. - When the merge/compression parameter window is present, you can select the compression mode using numeric keys, and increase/decrease the merge threshold using the "+" and "-" keys. You can cancel using ESC and proceed by hitting RETURN. - The deplode query requester reacts to the "Y" and "N" keys. ** Loading / Saving ** When you have specified a file for the Imploder to load, processing will commence (detailed below), and once finished the file requester pops up to query you for the destination. The source and destination paths are maintained in a special way that minimizes the chance of the destination not being what one wants. You could e.g. implode files from one directory and store them in another, or overwrite them in the source directory. In the first case it would be annoying if the destination directory defaults back to the source; you'd have to change it every time you want to write an imploded file. In the latter case, it'd be annoying to change to a different source directory and find out that the destination still points somewhere else. So the logic the Imploder uses is as follows; if you change to a different source directory, the destination directory will follow the source directory only if the destination is equal to the source. ** Batch Mode ** The Imploder can compress a series of executables by allowing you to specify multiple files by way of ?,#,* wildcards. All non-imploded executables matching the pattern will be processed. After entering the wildcards, the "merge and compression" parameter window will appear. The options present there will be explained below. The important thing to note here is that these selections will be global to all files processed during batch mode. Next, you'll be asked to specify a destination directory. This is were all batched executables will be saved after implosion. ---------- Processing ---------- Now you're familiar with the user interface, let me get on with explaining how the Imploder processes programs. The implosion processing sequence consists of four steps; -Loading & Format Verification. -Hunk Merging & Reloc Table Cleanup. -Implosion. -Decompression code installation, Overlay table adjustment and Saving. ** The Loader ** The loader reads the selected file. During this process, the executable file is analyzed. Format defects cause the loader to either report an error, and abort processing, or to give a warning while ignoring or repairing the defect. After having loaded a file, the "merge and compression" parameter window will appear. You can click the topright icon to collapse this window and review the status information produced by the loader. ** The Merger ** The merger is disabled by default. Most executables produced by modern compilers/linkers don't require this option, so if you're impatient you can skip to the next section. If selected, the merger will try to coalesce the hunks in a program to hunks of upto the merge threshold in size. If used in a constrained manner, this will reduce the size of the program, and the chance of memory fragmentation. If you specify a large merge threshold however, the resulting file will need large contiguous regions of memory in order to load, thus reducing the chance of it being able to be loaded into a fragmented system. Merging an executable file might break it. This is because certain programs make assumptions about the format of their segment list. The most notable members of this group are BCPL programs and a few libraries and devices. Hunk merging is therefore automatically disallowed for these. Still, some other programs, especially selfdetaching programs might dislike being merged. Regardless of whether a program has been merged or not, a reloc table cleanup routine is executed upon the file. This deletes empty reloc tables, coalesces matching reloc tables, and finally sorts the relocation offsets. This is of no consequence to how the program will look after it has been decompressed and relocated. ** Compression ** The compression algorithm can operate either in turbo mode or in normal mode. The normal mode requires no additional memory, whereas the turbo mode needs some 300K of additional hashing buffers. The turbo mode kicks-in automatically whenever the required amount of additional memory can be allocated. It is about ten to twenty times faster than the normal implosion mode. Note that the parameter window will report whether or not the current memory configuration allows the turbo to be enabled or not. You might try to free a few resources in order to make the target. Whenever you select a compression mode (gadgets 0-8), the turbo capability will be reevaluated. Various compression modes can be specified. These modes vary from zero (range = 128 bytes) to eight (range = 18K). The range related to each compression mode determines the maximum distance searched while looking for redundant data. The higher compression modes therefore have a better chance of compressing data. The time required to compress a program increases proportionally to the maximum distance in case of non-turbo operation. The processing speed during turbo operation however, is only slightly affected by varying the compression mode. So it only makes sense to fiddle with the compression mode if you don't have enough memory to have the Imploder run in turbo mode, and therefore want to speed things up (at the expense of a the compression efficiency). For all other instances it suffices to leave the compression mode at its maximum value (eight); for small executables the mode will automatically be scaled down to what the Imploder thinks is probably the most efficient one for the file at hand. During compression, the level meters to the right will display various relevant parameters. Going from left to right these are: 1. Percentage of program data still to be compressed. 2. New size of executable in % of the former size. Reevaluated continuously. 3. Skip. If a large chunk of non-compressable dat is encountered within a program, its level will start to rise. When it hits the top, the encoding limit has been exceeded. This is a rare occurance. 4. Length. Gives a measure of the redundancy of the data currently being processed. High levels indicate good compression. 5. Distance. Gives a measure of the non-locality of current redundancy. ** Decompression Code Installation ** All imploded executables require some additional memory in order to be able to decompress. The additional amount of memory required is about 50% of the original program size plus a constant amount of buffer space depending on the compression mode, and never larger than 20K. After decompression this memory is freed without causing memory fragmentation. Furthermore, the normal distribution of program-data across hunks is retained. This allows for the loading of imploded files into fragmented memory, and the proper distribution of hunks across chip and fast memory. The Imploder is able to install 3 different decompression algorithms into imploded executables; Library, Normal and Overlayed. LIBRARY IMPLODED files are the default, they are generated when; -The "Compress" gadget is enabled. -The "Library" gadget is enabled. -The processed executable isn't overlayed. To be able to use library Imploded files, copy the explode.library to your LIBS: directory. When you run a library imploded program, the decompression code in this library will be called. The library code is fast and removes the need of appending decompression code to every imploded file. The library has several special useful properties discussed in the "Library" document, but for simple use it suffices to make sure the library is in LIBS:. Pure programs may be library imploded. NORMAL IMPLODED files are generated when; -The "Compress" gadget is enabled. -The "Library" gadget is disabled (this is NOT the default). -The program being processed isn't overlayed. Normal imploded files have decompression code appended to them. A normal imploded program will run just like the original, and is independent; in contrast with library imploded files, normal imploded files don't require libraries or other special files to be present in your system. Disadvantages of normal Imploded files are that they are a couple of hundred bytes larger than library Imploded files, and their decompression speed is significantly slower. This is due to need for space optimizations in the decompression code. Once you normal-implode a pure program it's no longer pure. OVERLAYED IMPLODED files are generated when; -The "Compress" gadget is enabled. -the processed executable is overlayed. Overlayed programs are executables with additional appended hunks that are loaded during runtime. The Imploder will automatically recognize whether a program is overlayed. Overlayed files are rather loosely defined; basically the program is passed a bit of information that allows it to get at the appended hunks, but the means of doing so is left up to the programmer, though there is a standard technique specified by CBM which is used by the majority of overlayed executables. Because the Imploder decreases the size of the file, the offset of the overlay data also changes, and this must be corrected for. The Imploder knows how to do this for the standard overlay format. You'll be notified when the format isn't as the Imploder expects. Thus imploding overlayed files is not something which is guaranteed to succeed, so take heed; you should only implode overlayed files when you know what you are doing, and are willing to verify the results. --------- Deplosion --------- If you select an already imploded executable for processing (the ID gadget in the directory window allows you to see if files have already been imploded, at the cost of slower listing), you will be asked if you want to deplode it. If you confirm this query, the executable will be restored to a normal non-imploded format. Note that this format isn't necessarily bitwise identical to the original; the reloc tables are sorted, and you might have applied hunk merging (which is irreversible). Also, any symbol or debug hunks present in the original will have been stripped. Still, if you run the program, things will look the same to it, and this is ofcourse what counts. Note that there is a CLI based "Deplode" command available in the Tools directory which does basically the same thing. This is purely a matter of convenience ment for people that think of the CLI as convenient. *************************************************************************** The Imploder was conceived, written and performed by: Albert-Jan Brouwer - Programming, documentation. Peter Struijk - More programming, less documentation. Paul van der Valk - Music-programming and composition. Erwin Zwart - Graphics, aesthetic lay-out. Please direct queries, comments, bug reports, and other things that can not be resolved by rtfming to: UUCP hp4nl.nluug.nl!cbmnlux!ecl001!ajbrouw UUCP cbmvax.commodore.com!cbmehq!cbmnlux!ecl001!ajbrouw FIDO: 2:281/614 (up for freq sometime fall '91) Legal mush: The Imploder is Freely-Distributable, as opposed to Public Domain. Permission is given to freely distribute this program provided you include this documentation and other related files, and no fee is charged in excess of reasonable media and mailing costs. All programs in this distribution have been enforced and mungwalled. P.S. We've been promising a 4.0 version for over a year now, well here it is. Things got delayed slightly. We apologize for the inconvenience. *************************************************************************** Changes V3.0 -> V4.0 -------------------- Major: - Finally a complete and up-to-date well documented release with all of the support utilities. - Basic support for batchmode processing. - Radical new music composition by Paul van der Valk. - Erwin Zwart has completely redesigned the graphics to conform to the 2.0 look. Aesthetics and functionality have improved significantly. - Defaults to support for a "safe library root". This is larger startup code for library imploded files that warns when no library is present. You can override this setting. - The "Protect" option has been discontinued. See the "philosophy" document for the rationale on this. - Mouse hater support by way of a CLI based Deplode command, and keyboard equivalents throughout. Minor: - Support for "pure-imploded" files removed; If you know how to make a program resident, you know how to use the library. - The "Library" gadget now defaults to on. - The "Merge" gadget now defaults to off. Improved compilers/linkers plus the much more frequent appearance of selfdetaching programs have made this option less important and more dangerous respectively. - Added tooltypes and commandline switches for specifying the setup options. - The copperlist has been tuned down a bit, and the screen is now dragable. - The explode.library has been enhanced/sped up a bit. - Removed bug that caused Intuition to read from location zero. This would occasionally crash the machine when location zero was nonzero. This bug was already removed in V3.1 (thanks to Arnout). - All stuff has been enforced and mungwalled so now we've even better confirmation that no serious bugs are present. - Improved overlay file recognition. No longer expects overlayed files to have additional space in the hunk table. Searches for the magic longword. - Decodes ARP resident program tags. Warns against stack gobblers. - Data hunk ID now preserved upon deplosion. Inconsequential, yet neater. Very stringent format checks in the old deplosion routine barf on the extra bit required for this, but the old explode library doesn't mind, so in that sense it is backwards compatible. - Lots of other small changes. *************************************************************************** Introduction ============ The Imploder allows you to reduce the size of executable files while having them retain their full functionality. There are other "crunchers" or "packers" available for the Amiga, but none are as mindful of the complexities of your Amiga system as the Imploder (IMHO). In addition to this, its algorithms are more efficient, both in terms of speed, and size reduction. Basic operation is clicking and using pull-down menus. Though operating the Imploder isn't hard, the complexities of the Amiga give rise to the need for imploding different types of executable files in a number of different ways. Please note that not reading the documentation won't inhibit you from using the Imploder; it automatically detects, and reacts to almost all exceptions, and for the vast majority of files it suffices to simply hit the "Proceed" gadget after loading has finished. Still, the Imploder does have a few - quite useful - options that require you to have some understanding of the issues involved. Notably, it would be preferable for you to read the documentation before making use of the explode library or the merge option. ---- Judging from some of the reactions we've received, the old documentation didn't do a good job in communicating essential information to the user, so it has been revised and split into several parts for easy reference. Here's a list: - Authors; About the authors, legal status, distribution etc. - Changes; Highlights, newly implemented features and future intentions. - Manual; How to operate the Imploder. - Library; Information on the explode.library and related issues. - Techno; Some in-depth information on the Imploder's internals for those tech types that want to know it all. - Philosophy; This discusses the Imploder's design choices and intended use. *************************************************************************** ** Features ** Library-imploding files is preferable, not only because the result is smaller, but also because the library is faster, and able to patch the AmigaDOS executable file loader (LoadSeg), thus decompressing ANY type of load file as soon as it is loaded into memory, e.g. fonts, devices, libraries, etc. If you only library-implode program files, the library will be loaded when you run a library imploded program for the first time. However if you wish to implode non-program executables, you'll have to make sure the library is already resident because these won't load properly until LoadSeg has been wedged. If you're not yet using the 2.0 OS, there is a slight complication explained in the section at the end of this document. To make sure the library is resident, we've provided a small program that can be run at the head of your startup sequence. It's called "ExpLoad" and resides in the tools subdirectory. All it does is open the explode library. Once this is done, the library hooks itself into DOS and remains resident, even when a system flush occurs. Note that it suffices to run a library imploded program to make the library resident, but this is less visible so you might forget about it when you modify your startup-sequence later on. Once you've made sure the library is resident, you may merrily go around your system and library-implode libraries, devices, fonts and even keymaps! By default the Imploder appends a foolproof piece of startup code to library imploded files. This checks whether the library is present. If not it will print an error. If started from the WorkBench a tiny window will be opened to display the error message. However this error checking takes up a bit of space, and is redundant when you make sure the library is resident. For this reason you can specify the SHORTROOT tooltype/switch. When set, the Imploder will append only short startup code to library imploded files. This saves about 250 bytes for every executable. If library imploded programs with a short root do not find the explode library they will keep on trying to open it until they succeed. These programs will therefore hang until the LIBS: directory contains the library. Still, using the SHORTROOT switch has its advantages. The reason it is not enabled by default is to make sure that people have read this document first. ** Pre-2.0 problems ** If you are NOT using 2.0 or some newer OS, there are a few additional problems related to BCPL braindeadness, so if you're still running 1.3 or less, READ THIS! Under a pre 2.0 OS, if you run a library imploded program generated with the SHORTROOT option from the WorkBench, while the library hasn't been made resident yet, the system will guru. This is a problem caused by the OS. The default library startup code checks for this condition, and will exit cleanly after reporting an error. So under 1.3 or less it is advisable to make the explode.library resident before the WorkBench is active, e.g. early in your startup sequence, even when you're using library imploded non-program executables; The first invocation of a library imploded program might happen from the WorkBench. In addition to this it is best not to library implode handlers under pre 2.0. See the Techno document for an in depth explanation of these issues. ** Conclusion ** Library implosion limits the possibility of passing on one's executables to other people/systems who/that do not have the explode.library installed. I think this doesn't really matter because I strongly believe people should be left the option of deciding in what way, if at all, they are going to compress/install executables. (See the "Philosophy" document for a more extensive discussion of this issue). Evidently, using library imploded files is the most transparent to the system. We personally recommend using mostly library implosion, and only in special cases normal implosion, or overlay implosion for the odd overlayed file one might encounter. *************************************************************************** This doc may sound slightly grumpy in places, still this is how we feel about things, so there ;-) Intended use ------------ The Imploder is intended for creating extra space on your system disk while maintaining full functionality. To achieve this goal, we've tried to make the decompression process as invisible and fast as possible. Thus the Imploder doesn't support any annoying colour flashing, and it has a high speed of decompression. On a vanilla 68000 Amiga, "explosion" speed is about 30-50 K/s, depending on the type of compressed code. Also, the explode.library is a bit faster than the explosion routines appended to stand-alone "imploded" files. You should take this speed issue into consideration when determining which executables to implode. For floppy users, the startup times will mostly be FASTER. Users with fast harddrives however might want to limit themselves to imploding only infrequently used executables. It is therefore very useful to floppy users, yet harddrive users can also add a few megs to their bit budget. Evidently, the optimum use of the Imploder will vary from system to system. A3000 owners for example won't even be able notice programs exploding. Because of this, over the years, the emphasis of the Imploder's intended use has moved away from Imploding being a once in a program's life time operation, suitable e.g. when an author wants to distribute his program. Instead we now feel that every user should be able - when installing a program - to decide whether or not to implode it, and in what way. The user's responsibility ------------------------- So the issue here is freedom of choice. And it is responsibility of the user to apply the Imploder in a sensible fashion. To get a feel for what I'm talking about simply look at what happened when people started distributing Power-Packed text files. Many people didn't like using the PPMore reader or simply didn't have it. This kind of thing can be utterly annoying, and the Imploder has the potential for the same kind of abuse. We therefore recommend limiting the use of the Imploder to compressing things installed in your system. So if you must pass on a program to someone else, use the original archive. This keeps everyone happy, including the program's author. Another point is that when receiving programs from PD sources, people should be able to easily check if programs contain file viri or other unwanted things. If an executable is imploded it'll have to be deploded before one is able to examine the code. So don't distribute compressed executables of _any_ type. Still, there was one action we could take to encourage the kind of use we recommend; In previous versions one could select a "protect" option that prevented a program from being deplodable. The intent of this option was to protect an author's work before he distributed it. In reality people started using it to protect dirty hacks or programs containing file viri. Evidently this is incompatible with our philosophy. Thus, the support for protecting programs has been discontinued as of version 4.0. In addition to this, the Imploder now allows decompression of any protected files left over from the past. *************************************************************************** This document deals with the inner workings. If you're a tech type, and are interested in somewhat deeper lying aspects relating to how the Imploder operates, chances are you'll find it here. Subjects covered are: - Compression - Decompression - Reversibility - The Library - Overlayed Files - Merging - ARP - The Music - The Copperlist - 68040 Cache Coherency ** Compression ** The Imploder (we only recently learned :-) does LZ77 like compression with a per-mode static Huffman coding step on the various parts of the skip, offset and length tuples. Due to the efficient encoding, a tuple can require less than 12 bits, and thus strings of 2 bytes length are encodable with a decent gain (given small Huffman patterns corresponding to likely circumstances). To speed up the string searching, the turbo mode causes the accessible strings to be indexed using a hashing table. However, the fact that strings with a minimum size of two bytes are still potential candidates for compression, requires the hashing function to necessarily be rather simplistic. When the implosion algorithm processes highly redundant data, entries in the hashing table tends to get very imbalanced, causing a reduction in performance. For most types of data, notably executables, this isn't the case though. ** Decompression ** The goal of decompression is to reproduce the segment list of the original program. This is a linked list of data/code/bss hunks, some of which require being positioned in chip memory. The decompression code will have to fill the data and code hunks with the decompressed data, and, if required, perform relocation of absolute references. The Imploder lets LoadSeg allocate all target hunks (as BSS). These are at the start of the hunk list. This is followed by a hunk with the decompression code (only for non-library imploded programs), a compressed data hunk (normally about 50% of the static data size, depending on compression gain ofcourse), and a decompression buffer (only upto 17K in size). As you can see, no allocations need to be done, so the exploding process will never fail. During decompression time, data is decompressed from the compressed data buffer into the decompression buffer until it has filled. It then empties this buffer by filling hunks and processing reloc tables. When the buffer has been emptied, the process repeats until all data has been scatter-decompressed across the target hunks. Now you might ask, why not directly decompress to the target hunks instead of using a decompression buffer? This is because the the Imploder uses the decompressed data to decompress, and needs to be able to easily access it (for speed). Referencing data distributed across several hunks is much more cumbersome. An added bonus of having separate source and target memory regions is that a notation can be used that doesn't gain under rare circumstances, but on average yields better results (No chance of source/target pointer collision). When explosion has finished, the decompression buffer, code and data hunks are freed, and memory usage reduced to what it would have been for a non-imploded version of the program. People often compare the Imploder to the Power Packer. The Imploder decompresses faster and looks cooler [:-)], but the most interesting differences lie in the implementation of the various steps of the decompression proccess. So let's contrast the advantages of the Imploder's approach with the Power-Packer's implementation. - By having LoadSeg allocate all memory as BBS hunks, explosion will never fail. The Power-Packer on the other hand allocates hunks. If it fails it will simply exit. Power Packed programs launched from the WorkBench thus won't reply the startup message, which will be left dangling in memory forever. - Memory doesn't get fragmented. The explosion related hunks are at the end of the seglist and thus were allocated (by LoadSeg) AFTER the target hunks. This isn't true for the Power-Packer. It does leave a hole in your free memory list when it frees its decompression stuff. - Additional memory usage is only about 50% of the static data size + the size of the decompression buffer, which is always small relative to large programs (maximum 17k). So a 30K program might require 62K to decompress (30+15+17), a 300K program will require 467K (300+150+17), assuming a 50% compression reduction. The memory usage report generated after a program has been imploded includes BSS hunks. I've discussed only static data here. BSS hunks don't require any extra memory usage of course. Power-Packed files require a buffer as large as the original program for both compressed data storage and decompression. Memory usage is therefore always about twice the static data size (again ignoring BSS) while for the Imploder it drops to 1.5 for executables large enough to matter memory wise. (In this comparison I'm talking about executables as produced by Power-Packer version 3.0b.) Non-library imploded programs have a small first hunk that calls the decompression code hunk at the end, and frees these last three hunks. For library imploded programs this freeing occurs in the library, so no preceding hunk is needed. ** Reversibility ** Before compression the Imploder preprocesses executables. It kicks out all the redundant stuff by merging subreloc tables referring to the same hunk, sorting relocs (improves compression), removing debug symbols etc. etc. This is what all those info blurbs in the text window are about. So the deploded executable isn't guaranteed to be byte by byte identical as far as loadfile control codes are concerned. What is guaranteed is that the memory images created when the original, imploded and deploded program versions are loaded are identical. So the deplosion process isn't 100% reversible. Normally this is no problem. The reason for uncrunching is mostly wanting to recompress an executable using a different compression mode, or having a quick peek at the code e.g. when applying a patch with something like NewZap. If however you expect to need the debug symbols, or (important) require the executable to be in the _exact_ original format in order to have things like lpatch updates to applied, you're out of luck if you've only got the compressed executable. So always keep the original archives/disks! This is yet another argument for retaining the original archives. The Imploder is an online space creator not a distribution archiver (See the "Philosophy" text). ** The Library ** The library code has been unrolled a bit, and optimized here and there in order to achieve optimal performance. This makes it faster than the normal explosion algorithm. If you library implode a program there is NO way in which the program, after explosion, will be able to notice. If you make sure the library is resident, this is also true for any executable file loaded for any purpose by any program. For normal etc. imploded programs the startup/cleanup hunk mentioned at the end of the "decompression" section might be detected if a program goes through contortions involving finding the segment list via murky DOS structures instead of simply doing PC relative hunk start referencing which also works from the WorkBench. I haven't encountered any programs that do this. Still this is yet another reason to use the library; there is not even the slightest chance of it being incompatible with an executable. Note that the Loadseg vector is patched in an "intelligent" manner; it will install fine for pre 2.0 kickstarts (braindead jumptable format) as well as in BCPL free systems (2.0+) Under pre 2.0, when a library imploded file is run from the WorkBench, and the explode.library isn't resident yet, Exec will try to load the library from disk. The process's message port however is in use by the WorkBench reply message, and until it has been replied, it cannot be used by the DOS in order to send packets. Thus the DOS gurus. Also, BCPL code doesn't jump through the the library vector. The only structural problem with this are handlers. These are loaded by the DOS, and the DOS is BCPL code, again ONLY under < 2.0. Under 2.0 the library works just like intended when it was first conceived. Transparently that is. ** Overlayed Files ** The Imploder compresses the load part of an overlayed file as if it were a normal executable file. Subsequently, the overlay table and the overlayed program section are appended. It then tries to adapt the overlay table. Because different types of overlay supervisors are in use, the format of the overlay table isn't known to the Imploder. The only assumption made is that the overlay table contains seek-offset longwords, at longword aligned locations, that point into the file to the hunk_header ($3F3) identifiers of the overlayed program sections. This is how the standard overlay manager operates, but nothing prevents a programmer with sufficient technical knowledge to create a novel overlay format (e.g. selfextracting DImp files). If the Imploder finds one of these offsets, it is adjusted by the amount the initial load part of the executable file has compressed. The deplosion algorithm also tries to find these offsets when restoring the overlay table. Thus there is always a very small chance that the imploder will adapt a longword that was never meant to be an offset. An overlayed file gets its information from the loader in four longwords, at the start of the first hunk. In an imploded overlayed file, this hunk is the root hunk, and after decrunching these longwords are adjusted and moved into the first hunk of the actual program (the second hunk of the seglist). Evidently this process can never be 100% deterministic, so take heed and test any overlayed programs you've Imploded. Or don't use overlay implosion at all if you can spare the bits. ** Merging ** Though modern linkers/compilers typically produce executables with one code hunk and one data hunk, there are still some old executables and less evolved linkers around. The merging option was implemented when executables with sufficient hunks to cause a lot of redundancy were still commonplace. Every hunk requires a longword in the allocation header, plus a hunk ID, load size, and hunk end ID. That's 16 bytes per hunk, and thus saved for every merge action. Doesn't sound like much, but generally hunks also have reloc tables. These waste a lot more space, especially with references to a lot of different hunks, though there's no easy equation. The merging step merges matching hunks (data-data, chipdata-chipdata, code-code) into hunks of upto the merge threshold in size. The actual size is of course determined by the sum of the sizes of the composite hunks, and may very well be a bit less than the specified threshold. Obviously this process discards redundant data in an irreversible fashion, so deploding the executable won't reverse it. Lots of tiny hunks cause memory fragmentation, but increase the chance of the program being able to load when the system is fragmented, and low on memory. Thus there is a kind of optimal balance that varies from system to system. In general it can be said that hunks less than 10K or more than 100K are "bad". Another factor is that loading a program with many tiny hunks causes the LoadSeg function to issue double as many tiny read commands, thus bogging down the speed with which an executable can be loaded into memory. For simplicity's sake, I've chosen for the Imploder to process executables within a single buffer, without the need for additional backup buffers. Thus, removing redundant information, and copying hunk data during the merging and reloc cleanup process involves moving or mirroring large parts of the buffer. This is why merging can take a while when processing a large executable with a hundred or so hunks. ** ARP ** Programs written for use with the ARP shell are able to specify the stacksize they require inside the first hunk of their executable. If such a program is normal or pure imploded, the segment list won't become visible until the program is run. Thus ARP has no way of finding out what the proper stacksize should be. Library imploded programs have no trouble with this because they are already decrunched after they are loaded into memory with LoadSeg. (Provided the library has already been made resident.) The Imploder will recognize these files and report on them. If the requested stack-size is larger than the usual minimum (4000 bytes) a warning will be printed, and you'll be urged to use only library implosion. The chance of a programmer relying on a soon to be obsolete shell for setting a stack LARGER than the usual default is rather slim though. It would have been very nice if 2.0 had sported such a stack setting feature, and indeed it had been planned, but was never implemented due to lack of time on the part of the Commodore programmers. We'll be on the lookout for any future changes to the executable file format in order to fix any potential incompatibilities before they'll cause problems. ** The Music ** When we got word the CIA-A timer was used by the OS under 2.0, we switched to trying to allocate first CIA-A and if not available CIA-B to drive the music. However the CIA-B interrupt priority is too high and can interfere with things like serial transfer. So Paul got this great idea to keep on using a CIA for precision timing purposes, but drop down to a SoftInt for doing the actual work, modulations, etc. This works great, the amount of code executed under CIA priority is now negligible. Recently, the CATS started feeling guilty about hijacking the CIA-A timer and thus created "Jumpy the magic timer device". If I understood things correctly the latest 2.0 timer device moves out of the way and starts using a less accurate timing source whenever an application tries to allocate the CIA-A. Pauls music driver can run of both CIA-A and CIA-B, and it would be a pity to make Jumpy jump without good reason, so he changed the alloction sequence from A-B to B-A. ** The Copperlist ** There are a couple of unavoidable quirks when one uses copperlists on Intuition screens. On certain machines, probably PAL, under certain circumstances, dragging a dense copperlist past scanline 256 or so will cause some video crud to appear at the top of the display. This can't hurt, but it sure does look ugly. I suspect this is a hardware misfeature because it ain't fixed yet under 2.0 This was the reason why the screen of older Imploder versions wasn't draggable; you might just think this muck was our doing. Second problem is that copperlists "shine through" onto other screens in front. For this reason we've choosen a colour > 4 for the level bars, so this is never observable with screens less than 3 bitplanes deep. The 2.0 OS support proper copperlist clipping, but it has been disabled by default for compatibility reasons (yeach). Supposedly there is a bit somewhere in the graphics base to turn this back on, so I'm sure, in due time, there will be some preference editor to re-enable this. ** 68040 Cache Coherency ** With the advent of the 68040 processor, programs that diddle with code which is subsequently executed will be prone to some problems. I don't mean the usual self-modifying code causing the code cached in the data cache to no longer be as the algorithm expects. This is something the Imploder never had a problem with, indeed the Imploder has always worked fine with anything upto and including an 68030. The reason the 68040 is different is that it has a "copyback" mode. In this mode (which WILL be used by people because it increases speed dramatically) writes get cached and aren't guaranteed to be written out to main memory immediately. Thus 4 subsequent byte writes will require only one longword main memory write access. Now you might have heard that the 68040 does bus-snooping. The odd thing is that it doesn't snoop the internal cache buses! Thus if you stuff some code into memory and try to execute it, chances are some of it will still be in the data cache. The code cache won't know about this and won't be notified when it caches from main memory those locations which do not yet contain code still to be written out from the data caches. This problem is amplified by the absolutely huge size of the caches. So programs that move code, like the explosion algorithms, need to do a cache flush after being done. As of version 4.0, the appended decompression algorithms as well as the explode.library flush the cache, but only onder OS 2.0. The reason for this is that only OS 2.0 has calls for cache-flushing. This is yet another reason not to distribute imploded programs; they might just cross the path of a proud '40 owner still running under 1.3. It will be interesting to see how many other applications will run into trouble once the '40 comes into common use among Amiga owners. The problem explained above is something that could not have been easily anticipated by developers. It is known that the startup code shipped with certain compilers does copy bits of code, so it might very well be a large problem. *************************************************************************** Deplode is a CLI command that allows you to quickly "deplode" imploded executables. Usage: Deplode <source> [target] If no target filename is specified, the sourcefile will be overwritten with its deploded counterpart. *************************************************************************** Disk Imploder V2.27 by A.J.Brouwer. FreeWare --------------------------------------------- NAME DImp - Archives cylinders. SYNOPSIS DImp READ <SourceDrive> Ca[-Cb,Cc,..] <TargetFile> [-M?|-NB|-NC|-X] [+Readme] The collection of cylinders to be read must be specified by ranges, Ca-Cb, and/or single cylinders, separated by commas. -M? or MODE? = Compression mode, ? = 0-7, default = 5 -NB or NOBITMAP = Ignore bitmap (normally unused blocks are cleared) -NC or NOCOMP = Do not compress cylinders -X or EXE = Create self-extracting overlayed executable +"Name of Readme file to be displayed during WRITE" DImp WRITE <DImpFile> <TargetDrive> DImp INFO <DImpFile> DImp ABOUT DImp will use ".dmp" or ".dex" filename extensions when trying to create or recognize DImp data files. DESCRIPTION DImp allows you to compress a subselection of cylinders from any floppy compatible device into a file. DImp is fast and sports a good compression ratio. Notably, DImp allows you to create a self extracting file with an appended readme text, thus providing you with a completely self-contained method of distributing a disk's contents in a file. There are four operation mode keywords; "READ" specifies that cylinders must be read, "WRITE" indicates that you wish to write cylinders encoded in a dimped file, "INFO" displays information about the contents of a file containing dimped cylinders, and "ABOUT" should be obvious. The cylinders are individually compressed, checksummed and stored, thus, in case of a transmission error, valid cylinders can be undimped, and the originator can then select, dimp and retransmit the faulty cylinders. The cylinders to be READ can be selected by entering cylinder numbers separated by commas, or a dash if you wish to specify a range of cylinders. When reading or writing, DImp can handle any device that supports the normal floppy 2 heads 11 sectors etc. layout, thus it is usable with f.e a RAD: or an FMS disk. You can omit the colon normally following these device names. Additional options include compression mode selection override. Given the fact that the tracks are compressed individually, the default mode 5 has a pretty optimal compression ratio, though the ratio isn't as good as it might have been if cylinder data had been treated as a continuous stream. This is a design choice intended to allow the aforementioned flexible handling of individual cylinders. If there's sufficient free memory, DImp will allocate buffers for a turbo compression algorithm. Also, reducing the crunch mode will speed things up, dramatically so if you don't have sufficient free memory for DImp to run in turbo mode. Lowering the crunch modes results in less efficient compression though. Next, there are a couple of option switches. NOBITMAP prevents DImp from discarding data that, according to the disk's bitmap, isn't used by the filing system. This option is rarely needed. If you suspect a disk to contain a program that uses a low level routine to read its information from that disk, this option might be required to prevent loss of relevant data. NOCOMP disables compression. EXE causes DImp to generate self extracting dimp files. These can be executed, and will write their contents to a disk selected by the user. Executable DImp files (.dex filename extension) can be treated as normal DImp files (.dmp filename extension) in that you can use DImp to WRITE these, or get INFO on them. Lastly, you may specify a filename containing text to be displayed during the writing or self-extraction of the dimp file you're creating. This allows you to include a fairly extensive comment on the contents of the disk. This readme filename must follow a "+" commandline indicator. EXAMPLES DImp READ rad 0-79 Work:Cylinders EXE +ram:blahblah TECHNOCRAP The self-extracting executable files generated by DImp are overlayed. This allows for the cylinder data to be spooled during the self-extraction process, resulting in much more modest memory requirements than the size of the executable would lead one to suspect. Thanks to resource tracking, transparent error management, modular algorithm linking, and sitting on the program for several years before releasing it, DImp should be very robust and return sensible error messages. DImp is pure and can be made resident. AUTHOR Albert-Jan Brouwer UUCP hp4nl.nluug.nl!cbmnlux!ecl001!ajbrouw UUCP cbmvax.commodore.com!cbmehq!cbmnlux!ecl001!ajbrouw FIDO: 2:281/614 (up for freq sometime fall '91) *************************************************************************** File Imploder V2.33 by A.J. Brouwer =================================== This program is Freely-Distributable, as opposed to Public Domain. Permission is given to freely distribute this program provided you include this documentation and any other related files, and no fee is charged in excess of reasonable media and mailing costs. Usage: FImp <Source> [Target] [-M?|MODE?] [-XO|EXPLODEONLY] [-IO|IMPLODEONLY] [-RF|RUNFORMAT "..%s.."] [-AL|ARGLIST "stctsct..."] [-V|VERBOSE] Modes range from 0 to 11. FImp is a compress/uncompress command that allows you to reduce the size of any file. The algorithms used are similar to those in The Imploder; they sport a good compression ratio and blazing decompression speed. You can be easily customize FImp to operate transparently on files in your environment. Operation --------- If you don't specify a target file, FImp will try to replace the file you specified, and will expect/create imploded files with a .im extension. Suppose you have a file "test" in the current directory, "FImp test" will cause it to be replaced with an imploded file "test.im", and typing "FImp test" once more will replace "test.im" with the exploded file "test". Alternatively you can specify a target file or directory. In the case of a complete target file specification, optionally with preappended path, FImp will use both the source and target without any modifications. If you specify only a target directory, the filename part of the source will be used as a filename. This works fine because FImp examines the file data to see whether a file is imploded, and thus can determine if it should proceed with imploding or exploding. Note that the ".im" entension is only relevant when you don't specify a target. If you do, it will be neither appended while imploding, nor removed when exploding to a directory. This means that it's up to you to specify the ".im" extension for the target if you prefer to recognize the file type by way of the file name. Both FInf and FImp have support for recognizing a file as having been fimped by examining its contents, so if you use batchfiles or pipe aliases to interface with whatever you've fimped there's no need to use the ".im" extension. In any case, you can have it both ways. The NICE switch causes FImp to operate with a task priority of one count lower than the one it was invoked with. The old priority will be restored upon exit. Thus FImp will only eat the spare cycles without bogging down the system. If you use the VERBOSE switch, FImp will give a progress report while compressing, and report on current activities like loading, exploding etc. Compression ----------- FImp doesn't (yet) support streaming compression/decompression; the file to be compressed needs to be loaded into a memory buffer. As is the case with The Imploder, a special turbo compression mode kicks in when you've about 300K memory left (more for higher compression modes). This will drastically speed up the implosion process. Decompression also requires a contiguous memory buffer as large as the decompressed file, but doesn't require any memory in addition to that. Before decompression, FImp performs a quick checksum to make sure the file hasn't been corrupted. By default what's most probably the optimal compression mode will automatically be selected by FImp. If you want to override this, you may specify the mode yourself using the "mode?" switches. Large compression modes normally have a higher gain, but take more time, and, in case of turbo compression, use up a lot of memory. The speed differences between the various modes are quite large if no turbo mode could be initiated due to lack of memory. The large compression modes require quite a bit of memory in order to be able to execute. If you have some free memory (>300K) but not a whole lot, and yet want to ensure operation in turbo mode, the best course of action is to override the automatic compression mode selection with a relatively small value, e.g. MODE5. Advanced Features ----------------- FImp has a number of features that allow you to easily apply it to customized tasks such as transparently decompressing imploded text files and loading them into your editor. The "ImplodeOnly" and "ExplodeOnly" switches allow only compression or decompression respectively. Normally FImp decides by itself by looking whether the file is already imploded. Beyond the "RunFormat" keyword you may specify a command to be executed when FImp has completed its operation. This isn't very useful unless you can in some way pass on the files just processed by FImp. Therefore this "RunFormat" string allows PrintF like formatting; Any "%s" format statements present in the string will be replaced by a filename. However if you use %s statements you'll also have to specify which filename it should be replaced with. That's what the ArgList keyword is all about. Beyond it you may specify a list containing only the characters "s" and/or "c" or "t". "c" and "t" are mutually exclusive. The "s" stands for source, the "t" for target, and "c" for conditional. So the ArgList string "st" will replace the first occurance of %s in the RunFormat string with the source path+filename, and the second occurance of %s with the target path+filename. Note that if you specify only a destination directory, a "t" will still cause replacment by both the path and an appended filename derived from the source file specification. Now what's this conditional thing? Well, if you don't know for sure whether a file has been imploded and e.g. only care about doing something with the exploded version, such as loading it into your editor, you can set the "ExplodeOnly" switch to prevent the file from being imploded when it hasn't already been imploded. In this case you'd want FImp to also perform the RunFormat command (probably containing an invocation string for your editor) even though it didn't explode a thing. And you'd want the source file to be loaded into your editor (The one that FImp found out was already in a non-imploded state). So this is what the "c" does; It replaces its respective "%s" in the RunFormat string by the target file name if FImp exploded or imploded the file. Yet when the file couldn't be processed due to you setting the ExplodeOnly or ImplodeOnly switches, the source file name will be used. One last thing that needs to be mentioned is that the runformat string supports escaping by means of the \ escape character. Thus with \" you can include a quote in the runformat string, and \n inserts a line feed. Examples -------- "FImp data" Will implode the file "data" and replace it by data.im if the file "data" hasen't been imploded already. The same command will explode any file "data.im" if present and replace the exploded version called "data". This way you can toggle a file between its imploded and exploded state. "FImp source target -rf "Delete %s" -al s" This will implode or explode the file "source" to the file "target" (depending on what state the source file is in) and delete the source file, but only when the operation could be completed with success. The best thing to do is to define some aliases to hide the rather longish commandlines from view; Edx=FImp >NIL: [] T:Explode -XO -RF "Execute ExEdit %s" -AL "c" This will execute a batchfile called ExEdit. This could contain something like; <---"S:ExEdit" .key filepa/a Edit <filepa> Delete T:Explode/#? <--- Thus conditionally loading either the exploded target or the never- imploded-in-the-first-place source into an editor. The delete command subsequently purges any intermediate files from a subdirectory in T:. Instead of using a batchfile, you may separate commands in the RunFormat string by using \n line feeds. Still, batchfiles are more flexible because of e.g. the If/Endif/Skip flow control. FImp Fish:Contents Fish:Contents -IO If you don't want to bother with re-imploding files when they have been modified, you could add a list of these FImp statements to your CronTab, thus causing re-implosion during nighttime. As you can see, you may implode or explode a file to itself, but you need to explicitly specify an identical source and target. The original file will be deleted, and the new file written. This results in a "window of vulnerability"; during the write operation there is no backup of the file. The replace mode (when you don't specify a target), is safe in this sense. The source file will only be deleted after the target file has been successfully written. General Info ------------ Don't think of FImp as a general purpose archiver. Rather it is a quick and easy solution to compress single files. I don't endorse FImped files as a "public" format. There's enough confusion already. You might use FInf to "add" wildcarding capabilities to FImp, or to preselect only FImped files from a directory. The rationale behind all this is that by providing only the building blocks, documentation and some examples, the user retains the freedom to implement things as he/she sees fit. FImp stands out because of its decompression speed. If you'd like to compress external data used by a program you are writing, while hardly incurring any performance penalty, consider using FImp. If you query us, we'll provide you with the decompression routines in linkable, object format. Note that decompression happens in-situ and doesn't require any additional memory. If you wish to make a program that deals with "public" data, e.g. a text reader or a picture display program, you'd do best to wait for the xpk library by Urban Mueller. This is an interface library a la XPR to various compression algorithms. This way people will have an easy way of decompressing this "public" data without having to know about the various formats. FImp is pure and can be made resident. Enjoy. # AJ *************************************************************************** ; You can use this alias to read an imploded document. Alias EdExp FImp >NIL: [] T:Ex -XO -RF "Ed %s\nDelete T:Ex/#?" -AL C ; This does the same thing apart from using FInf to add wildcarding for ; loading multiple files. Alias Edx FInf >NIL: [] -q -p -f -na -nh -lf "FImp %s T:Ex -XO -AL c -RF \"Ed %%s\\nDelete T:Ex/#?\"" BATCH *************************************************************************** File INFo manual ================ File INFo V1.131 by Peter Struijk Introduction ------------ FInf is a very versatile directory listing utility. It can examine the contents of files and display a short type description. In addition to this, FInf has a whole slew of options that allow you to filter files by type, date, age, size etc., as well as recursive directory descending, and adjustable output formatting. So next to simply listing directories, FInf is extremely useful for creating hybrid commands that perform functions closely tuned to your specific needs. Command Line Arguments ---------------------- Usage: FInf [file/dir] [-lf=LFORMAT ["..%s.."]] [-nl=NOLINE [prefix]] [EXE] [IMP] [DOC] [IFF] [NOI] [-t=TYPE string] [-nt=NOTYPE] [-na=NOANSI] [-nh=NOHEAD] [NEWER [object]] [OLDER [object]] [-d=DIRS] [-f=FILES] [-q=QUICK] [-p=PATH] [-a=ALL] [NOX] [NOC] [BLOCK(S)] [DATES] [-lt=LONGTYPE] [SUB string] [BATCH] [LARGER size] [SMALLER size] [SINCE [n] date] [UPTO [n] date] [-ho=HEADONLY] Arguments surrounded by double-quotes are NOT recognized as keywords, this, or the use of -?? shortcuts can help you circumvent problems with identically named files and keywords. Keywords/switches are case independent. FInf parses its arguments in a left to right order, which may cause keywords to be overridden by keywords that occur later in the command line. E.g. "FInf -nt doc" displays only text files while "FInf doc -nt" will disable file type checking. You may use the backslash "\" as a fixed escape character; If you enter \n it will be replaced by a new-line, while \" implements a double-quote. FInf sets a returncode level of 20 (FAIL) on a serious error. If no errors occur, yet no matching file can be found, a returncode 5 (WARN) will be set. Examples -------- In order to whet your appetite, I'll start with a few examples. A complete option reference is given later on. Typing "FInf" without any options will display the files in the current directory, e.g. EM 14228 --p-rwed 12-Jul-90 22:47:39 XLIB IMP TeX 130344 ----rwed 04-Feb-90 19:29:42 EXECUTABLE Introduction 1918 ----rwed 21-Jul-90 15:23:12 ASCII TEXT Read Me 1532 ----rwed 22-Jul-90 16:31:52 ASCII TEXT Graph 4526 ----rwed 01-Jun-90 18:13:47 IFF PIC Tmp (dir) 5 files - 1 directory - 307 blocks - 152548 bytes Typing "FInf doc" will display all text files in the current directory, and "FInf disk* exe" will show all executables starting with "disk". If you use a shell that supports unnamed pipes and aliases, you can very easily construct new commands as shown below. If you do not have unnamed pipes, you can use the BATCH option, the PIPE: device or temporary files to obtain equivalent though less elegant functionality. The BATCH option will cause FInf to output to a temporary file and execute the contents like a batchfile once it's done. This will give you the same functionality as single level unnamed pipes. ------ E=FInf [] noi doc path files lformat " \"%s\"" noline ED | execute or E=FInf [] noi doc path files lformat " \"%s\"" noline ED batch This alias will execute the command ED with a list of text files that match any wildcards you specify. Typing "E" while in the directory shown above would have executed the command `Ed "Introduction" "Read Me"'. Note how escaping the quote character allows you to put quotes around the produced filename, thus avoiding problems with filenames that have spaces in them. Do not be deterred by the numerous parameters. Aliases need to be constructed only once, and when you are examining the requirements of a new alias, you'll almost automatically conclude you're in need of certain options. Chances are FInf provides them. ------ find=FInf NOTYPE FILES QUICK NOHEAD NOANSI PATH ALL SUB [] or abbreviated: find=FInf -nt -f -q -nh -na -p -a SUB [] This alias will search your current directory, and all subdirectories for filenames that contain a string you specify. You can search any directory by specifying a directory name *after* the search string. Very useful for finding files on your harddrive. If a matching file is found, the full path will be displayed. ------ FInf *.c Quick Path All | Zoo aI <archive> The I option in Zoo causes it to read filenames from the standard input, and these are provided by FInf. This particular example will archive all C source files in the current directory and its subdirectories. File Type Filtering Options --------------------------- FInf has a set of command line switches that causes it to display only specific file types. Here's a list: -t=TYPE <string> When FInf recognizes a file, it is able to print a string describing its type. The TYPE option allows you to display only those files where the pattern <string> occurs in this file type description. This keyword can be used to select virtuallly any file type or groups of file types just by supplying a carefully chosen <string>. Note that the LONGTYPE option causes FInf to search the extended file type description instead of searching the short description. List of currently recognized file types: The following list can be generated by "FInf ?" followed by another "?" at the ":" prompt. FInf's file type recognition is not by any means complete and even a bit outdated but serves well to distinguish between major groups of files like e.g. executables, text, iff. SHORT LONG UNREADABLE "Unable to read" UNKNOWN "Unknown type" FIMP DATA "FImp data file" OBJ CODE "Linkable object code" TTW PIC "TTW picture" PW DATA "PowerWindows data" DIMP DATA "DImp data file" IFF PIC "IFF picture" IFF SOUND "IFF sound" IFF SCORE "IFF score" IFF TEXT "IFF text" IFF DMS "IFF Deluxe Music score" IFF ANI "IFF animation" IFF S3D "IFF Sculpt 3D scene" IFF DLV "IFF Deluxe Video" IFF DOC "IFF document" IFF FILE "IFF data file" ASCII TEXT "ASCII Text" OLD IMP "Old imploded" NORM IMP "Normal imploded" PURE IMP "Pure imploded" OVLY IMP "Overlayed imploded" XLIB IMP "Library imploded" XLIB IMP! "Short library imploded" EXECUTABLE "Executable" OVERLAYED "Overlayed executable" NORM IMP* "Protected normal imploded" PURE IMP* "Protected pure imploded" OVLY IMP* "Protected overlayed imploded" XLIB IMP* "Protected library imploded" FONT HDR "Font header" WB ICON "WB icon" DISK ICON "WB disk icon" DRAWER "WB drawer icon" TOOL "WB tool icon" PROJECT "WB project icon" GARBAGE "WB garbage icon" DEVS ICON "WB device icon" KICK ICON "WB kick icon" In addition to filtering file types with the TYPE option there are a number of often used types that may be directly specified by keyword. These are: EXE - Display executable files (except imploded executables). IMP - Display imploded files and imploded data files. DOC - Display ASCII text files. IFF - Display all kinds of IFF files. You may also use the TYPE keyword to accomplish further differentation. Other Filtering Options ----------------------- In addition to type filtering, FInf can also examine other properties with which to exempt certain files from being displayed: NEWER <object> Display only objects which were created after or at the same time as <object>. OLDER <object> Display only objects which were created before or at the same time as <object>. UPTO <date> Display only those objects created UPTO a specified date. UPTO accepts three formats: DD-MMM-YY TODAY/YESTERDAY/TOMORROW/MONDAY...SUNDAY/FUTURE [n] TICK(s)/SECOND(s)/MINUTE(s)/HOUR(s)/DAY(s)/WEEK(s)/MONTH(s)/YEAR(s) The integer [n] is optional, default is 1. The latter option is extremely handy to cleanup your news directory. E.g. FInf NEWS: -a -p -lf "Delete %s" UPTO 8 WEEKS BATCH Make sure that your system clock contains the correct time! :-) SINCE <date> Display only object since a specified date. For information on the date formats see UPTO. LARGER <integer> Display only files with a size smaller than or equal to <integer> bytes. SMALLER <integer> Display only files with a size smaller or equal than <integer> bytes. SUB <string> Display only objects containing at least one occurance of <string> in their objectname. NOI Do not display .info files. -d=DIRS Display directories only. -f=FILES Display only files. -a=ALL Recursively displays the contents of any subdirectories in addition to the files/dirs in the current directory. Wildcards in the path description are not supported (yet), so for now this option will cause FInf to recursively enter ALL encountered directories. Formatting Options ------------------ These switches and keywords control the way in which FInf generates its output: BATCH This will capture FInf's output into a uniquely named temporary file in the T: directory, and subsequently will execute the file as a batch file that will be deleted after completion. This option is handy if you don't have unnamed pipes and something like the ARP execute command which will execute any commands piped to it. BLOCK=BLOCKS This switch will cause FInf to display the number of blocks files occupy on disk instead of their lengths. Note that for devices using the fast filing system, this value will include any extension blocks. DATES This will cause FInf to print absolute dates instead of dates with "Yesterday" of "Sunday" in them. -lt=LONGTYPE If specified, FInf will generate listings with a more verbose type description. In order to create printing space for this, the date and protection flags will not be displayed. -nt=NOTYPE If you use this switch, the contents of files will not be examined. The type description will therefore not be displayed. The EXE, IMP, DOC, IFF etc. filters won't work if NOTYPE is set. NOTYPE will cause FInf to operate about twice as fast because of the decreased amount of work it has to do for each file. -na=NOANSI This surpresses the generation of ANSI codes used by FInf to select bold or italic font styles in the footer and such. -nh=NOHEAD Like with the normal List command, NoHead will stop FInf from generating headers/footers with additional information on the examined files. -ho=HEADONLY This option is included to quickly simulate the unix DU command. DU stands for Directory Usage, an example alias: du = finf -ho -a [] NOX This option removes extensions from the filenames. This might be handy when you only wish to pass the filename root into the LFormat output. E.g. FInf *.twiddle NOX LFormat "Rename %s.twiddle %s.twaddle" | execute changes all .twiddle extensions into .twaddle extensions. NOC This simple option tells FInf to NOT print any filenotes (comments) which might be attached to a file or directory. Note that when the QUICK option is in effect filenotes are already been ignored. -q=QUICK In quick mode, FInf will display only filenames and any paths. No lengths, flags, dates, comments or types will be printed, the type filter switches will still function though. These bare (path+)file names can be more readily used by programs you intend to feed FInf's output to. -p=PATH If set, a full path will be pasted in front of any displayed filenames. This option is useful e.g. in conjunction with the LFormat or ALL switches. -lf=LFORMAT <"...%s...%s..."> Works like List's LFormat option. Briefly, you can specify a string with a "%s" in it (up to 8 %s expansions are supported), and where FInf normally would have printed paths+file names, it displays the given string with the path + file at the %s location. Useful for pasting commands in front of filenames so you can pipe the output of FInf to a batchfile, or directly to execute (if your shell supports this). E.g. FInf LFORMAT "rename \"%s\" \"%s.doc\"" FILES DOC BATCH will append a ".doc" extension to all ASCII files found. Note: LFormat automatically causes the Quick, NoAnsi and Nohead switch modes to be selected. -nl=NOLINE [prefix] This causes FInf not to print linefeeds between filenames. If you use this together with a LFormat " %s" and a QUICK switch, you'll get a line with file names separated by spaces. In the prefix position you may optionally specify a string to be pasted in front of this line. This will probably be the name of a command which requires a list of filenames as its parameters. Note that this line will only be printed if any filenames matching your wildcard specification were found. E.g. FInf NOLINE "Ed " LFORMAT "\"%s\" " PATH FILES will produce something like: Ed "ram:file1" "ram:doc1" "ram:readme" General Information ------------------- This program is Freely-Distributable, as opposed to Public Domain. Permission is given to freely distribute this program provided you include this documentation and any other related files, and no fee is charged in excess of reasonable media and mailing costs. Currently FInf supports the #,? and * wild cards. OS 2.0 wildcarding will be implemented sometime in the future. Also high on the wish list is a SORT option switch. If, during normal operation, there is only one file matching the file/wildcard specification, and it is a program file, some additional information about the number of hunks and the memory usage will be displayed. FInf has a lot of switches, so there's a chance that you might want to examine a file or directory that has the same name as one of these switches. You can inform FInf that the object you specified isn't a switch by putting quotes "" around it. FInf is pure and can be made resident. Enjoy. *************************************************************************** ; This batchfile demonstrates how to use FInf and FImp in making a "man" online ; documentation batchfile. ; FInf is used for scanning the manual directories. The documentation files ; may be (but need not be) imploded using FImp. Directories are used to ; differentiate between the different reader programs required to display the ; various files. .key manus/a .bra { .ket } FInf Man:{manus} -q -p -f -nh -na -lf "FImp >NIL: %s T:Ex -XO -AL c -RF \"Ed %%s\\nDelete T:Ex/#?\"" BATCH FInf Man:ANSI/{manus} -q -p -f -nh -na -lf "FImp >NIL: %s T:Ex -XO -AL c -RF \"More %%s\\nWait 1 sec\\nDelete T:Ex/#?\"" BATCH FInf Man:IFF/{manus} -q -p -f -nh -na -lf "FImp >NIL: %s T:Ex -XO -AL c -RF \"Run Leggi %%s smart font topaz/8\\nWait 2 secs\\nDelete T:Ex/#?\"" BATCH ***************************************************************************